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REMARKS 

Claims 1-5 and 7-23 are pending. New claims 21-23 have been added to recite 
additional features of Applicant's invention. Reconsideration of the application is 
respectfully requested for the following reasons. 

On April 14, 2004, the Examiner graciously extended Applicant's 
representative an interview to discuss the rejections in the Office Action. During the 
interview, it was emphasized that (1) no teaching or suggestion was in existence at the 
time the claimed invention was made to combine the Ma, Liddy, and Herz references 
in the manner suggested by the Examiner, nor would one of ordinary skill in the art 
be motivated to do so because such a combination would render an inoperable result, 
and (2) even if the references were combined the resulting combination would not 
include the "whole feedback information" recited in the claims. 

At the conclusion of the interview, the Examiner agreed to postpone his 
decision concerning the patentability of the claims upon receiving this paper, which 
sets forth the reasons for traversal of the outstanding rejections. New claims 21-23 
have been added to provide a more specific description of how the feedback 
information of the claimed invention relates to the image information in the data 
structure. As discussed during the interview, the cited combination of references do 
not teach or suggest these features. 
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In the Office Action, claims 1-3 were rejected under 35 U.S.C. § 103(a) for 
being obvious over a Ma-Liddy-Herz combination. Applicant respectfully submits 
that this rejection is in error for the following reasons. 

Claim 1 broadly recites the embodiments of the invention disclosed in the 
specification. As acknowledged in the Office Action, Ma does not teach or suggest 
"whole feedback information based on the user relevance feedback obtained since 
formation of the data structure" as recited in claim 1. To make up for this deficiency, 
the Liddy patent was cited. 

The Office Action states that "whole feedback" reads on "relevance 
feedback" on a "periodic time interval on page 3." Moreover, the Office Action 
acknowledges that Liddy does not teach or suggest a multimedia data structure which 
stores user relevance feedback information since the time the data structure was 
formed as recited in claim 1. Furthermore, the Office Action states that Ma and 
Liddy combined still do not teach feedback obtained since formation of the data 
structure on page 4. Thus, it is submitted that Liddy adds very little if anything in 
terms of teaching or suggesting the features of claim 1 missing from Ma. 

To make up for these deficiencies, the Herz patent was cited. Herz discloses a 
system which allows users to access newspapers, advertisements, and investment- 
related information in electronic form. In rejecting claim 1, the Examiner relied on 
two portions of the Herz patent to provide the whole feedback information of the 
invention. 
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The first portion (column 17, lines 30-40) discloses a filtering system which 
searches through target objects in a database and then estimates a user's interest level 
in each object found. The interest level is estimated from relevance feedback entered 
by a user. This feedback is stored on a long-term basis in association with each 
object. No specific indication is given, however, as to what "long-term" means, e.g., 
long-term in the context of the Herz system may be just a couple of days or months. 
What is clear is that this portion of the Herz patent does not teach or suggest "whole 
feedback information based on user relevance feedback obtained since formation of 
the data structure" as recited in claim 1. 

The second portion (column 19, line 45 - column 20, line 10) is even less 
relevant. This portion discloses a specific application of the filtering system applied to 
tracking the performance of an investment. In the Office Action, the Examiner 
indicated that the phrase "since formation of the data structure" was interpreted as 
corresponding to user feedback provided for invention since it was owned from the 
year 1990. This feedback information relates to performance of an investment taken 
over a period of years 1990, 1991, 1992, 1993, 1994, and 1995, and thus does not in 
any way relate to a feature of an image as recited in claim 1 . 

Moreover, the feedback information obtained in both portions of Herz is not 
included in a data structure as recited in claim 1. Rather, the feedback information of 
Herz is merely stored in association with the target object (e.g,. image) itself found 
during a search, which does not constitute a data structure. 
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Because the Herz patent does not teach or suggest "whole feedback 
information" relating to a data structure for an image , it is respectfully submitted the 
Herz patent cannot make up for the deficiencies of Ma and Liddy. 

For at least these reasons, Applicant respectfully submits that a combination 
of Ma, Liddy, and Herz cannot render claim 1 or any of its dependent claims obvious. 

Also, Applicant brings to the Examiner's attention that under MPEP § 
2143.03, if modifying a primary reference to include features of a secondary reference 
would render the primary reference inoperable, the combination is improper for 
purposes of rejecting a claim under 35 U.S.C. § 103(a). This situation exists here. The 
Ma patent discloses a method for retrieving images form an image database. This 
system uses a correlation matrix which organizes data in a cluster configuration. 
Feedback information is used to arrange this data in appropriate clusters. 

The Liddy patent discloses a neural network which performs internet searches. 
This neural network is trained with a set of training documents. When relevance 
feedback is received, the set of training documents is modified and the weights of the 
neural network are updated. 

Modifying Ma to include the neural network of Liddy would render the 
resulting system completely inoperative. The correlation matrix is the processing 
entity in Ma that performs searches. These searches are performed using a cluster 
configuration of data. Making the modification proposed by the Examiner would 
require replacing the correlation matrix of Ma with the neural network of Liddy. The 
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Liddy patent does not teach or suggest that its neural network reconfigures or 
otherwise operates using a cluster configuration for purposes of performing searches 
based on feedback information. 

Adding Herz only makes this inoperability more apparent. The Herz system 
uses a filtering system which performs searches and stores feedback information with 
investment and other information in a database. Adding Herz would require 
modifying the neural network of Liddy to operate with the Herz filtering system. 
Neither Herz, nor Ma, nor Liddy teach or suggest how such a combination can be 
formed and at the same time produce an operable result. Such an operable result is 
not possible because these elements are not designed to operate with one another. 

Because no teaching or suggestion exists to combine Ma, Liddy, and Herz and 
because a combination of these patents would produce an inoperable system, it is 
respectfully submitted that the cited combination was formed as a result of hindsight, 
formed as a result of the Examiner picking and choosing features from different 
references and then combining them without any legitimate basis for doing so. The 
Federal Circuit and Board of Patent Appeals both agree that such a combination is 
improper for underlying an obviousness-type rejection. 

Claim 3 further recites that the whole feedback information is "represented by 
a weight value learned by the user relevance feedback given since formation of the 
data structure." The Ma, Liddy, and Herz patents do not teach or suggest whole 
feedback information of any kind, let alone one which is represented by the type of 
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weight value recited in claim 3. Applicant therefore submits that claim 3 is also 

allowable based on the features separately recited therein. 

The Examiner rejected claims 4 and 14-20 under 35 U.S.C § 103(a) as being 

unpatentable over Ma in view of Liddy and further in view of Herz as applied to 

claims 1-3 above and still further in view of Cohen, U.S. Patent No. 6,067,539. The 

rejection is respectfully traversed for the following reasons. 

Claim 4 depends from claim 1. In order to render claim 4 obvious, the Cohen 

patent must teach or suggest the type of whole feedback information recited in claim 

1 which is missing from Ma, Liddy, and Herz. 

Cohen discloses a browser for performing a key word search on internet web 
sites. During this search, Cohen matches a score for a retrieved message which 
reflects the relevancy and the reliability of the message. Cohen, however, does not 
teach or suggest a data structure including whole feedback information obtained since 
formation of the data structure. Absent a teaching or suggestion of these features, it is 
respectfully submitted that a Ma-Liddy-Herz-Cohen combination cannot render claim 
1 and thus claim 4 obvious. Application further submits that no teaching or 
suggestion exists for modifying a Ma-Liddy-Herz combination to include Cohen, as 
such a modification would be inoperable. 
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New claims 21-23 have been added to the application. 

Claim 21 recites, in part, that the whole feedback information provides "an 
indication of relevance of said information describing the at least one feature of the 
certain image." None of the cited references teach or suggest or suggests whole 
feedback information of this type, e.g., the feedback information in Herz relates to 
performance of an investment over a period of years and not to any feature 
associated with an image. 

Claim 22 recites that "at least one feature includes at least one of a color 
histogram, a texture histogram, and a dominant color of the image." See page 2 of the 
specification where these features are listed. None of the cited references teach or 
suggest or suggests whole feedback information of this type 

Claim 23 recites that "said information provides a statistical description of the 
image." This statistical description may be in the form of a color histogram or some 
other form. None of the cited references teach or suggest or suggests whole feedback 
information of this type. 

CONCLUSION 

In view of the foregoing amendments and remarks, it is respectfully submitted 
that the application is in condition for allowance. If the Examiner believes that any 
additional changes would place the application in better condition for allowance, the 
Examiner is invited to contact the undersigned attorney, Samuel W. Ntiros T at the 
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telephone number listed below. Favorable consideration and prompt allowance are 
earnesdy solicited. 

To the extent necessary, a petition for an extension of time under 37 C.F.R. 

1.136 is hereby made. Please charge any shortage in fees due in connection with the 

filing of this, concurrent and future replies, including extension of time fees, to 

Deposit Account 16-0607 and please credit any excess fees to such deposit account. 

Respectfully submitted, 
FLESHNER & KIM, LLP 



Carl R. Wesolowski 
Registration No. 40,372 

Samuel W. Ntiros 
Registration No. 39,318 



P.O. Box 221200 
Chantilly, Virginia 20153-1200 

703 766-3701 DYK:CRW:SWN:JHB/par 

Date: April 29, 2004 

Please dir ect all correspondence to Customer Number 34610 



17 



